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ABSTRACT 



The effects of classroom assessment environment (CAE) 



variables on yearly achievement in mathematics and science in grades 
7 through 12 were studied through the Longitudinal Study of American 
Youth (LSAY) , using cohorts of approximately 3,000 students each in 
the 7th and 10th grades who were followed for 4 years. While the LSAY 
did not include all CAE variables of theoretical interest, it did 
include reasonable approximations of many. The direction of CAE 
effects was consistent for some variables and inconsistent for 
others. Percent of homework corrected and returned to students had a 
negative effect whenever it appeared. Hours of homework assigned and 
percent of students completing homework on time had positive effects 
when they appeared. Percent of class time used for testing and 
evaluation had significant effects only on math achievement (two 
negative and one positive). Frequency of oral reports had a 
significant negative effect for science, while frequency of written 
reports had mixed effects. Results do show some CAE effects, and CAE 
theory does explain some, but not all, of f.iese effects. Results 
support the beginnings of a theory explaining the relationship 
between classroom assessment and student achievement. Six tables and 
two figures present study data. (Contains 14 references.) (SLD) 
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EFFECTS OF THE CLASSROOM ASSESSMENT ENVIRONMENT ON ACHIEVEMENT 
IN MATHEMATICS AND SCIENCE 



Stiggins and Conklin (1 992) defined the classroom assessment environment as an 
overarching construction for how a teacher uses classroom assessment, including the part 
it plays in classroom instruction and student learning. This broad construct subsumes both 
objective indicators (e.g., frequency of testing), and subjective indicators (e.g., quality of 
tests). Th8 classroom assessment environment is not formed only by tests, but 
encompasses all the exercises and opportunities teachers arrange to observe and judge 
students and, sometimes, for students to assess each other. Stiggins and Conklin (1 992) 
developed the Classroom Assessment Environment Profile to measure this construct. Its 
dimensions include assessment purposes, assessment methodologies, criteria for 
selection, quality of assessments, feedback, the teacher as assessor, the teacher's 
perception of students, and the assessment-policy environment. 

One key to effective use of classroom assessment is the quality and usefulness of 
feedback to students (Bangert-Drowns, Kulik, Kulik, & Morgan, 1991). Students' 
perceptions of both the quality of the feedback and their own competence as learners 
depend in part on the feedback providing clear information students can use (Brookhart, 
1994; Ryan, Connell, & Deci, 1985). Student perceptions are important mediators of 
classroom assessment environment effects. Student perceptions of the quality of the class, 
its importance and utility, and how difficult it was for them were included as variables in this 
study. The theoretical argument for how these perceptions function to affect the way 
students interpret and use classroom assessment information is made in Brookhart (1995, 
available from author). 

How important is the classroom assessment environment to achievement? While 
it should be a major factor within classes (Stiggins, 1994), is it major when compared with 
known influences on math and science achievement like socioeconomic status (Cakes, 
1990; Walberg, 1991), gender (Tittle, 1986), or general academic aptitude? Understanding 
the effect of the classroom assessment environment, especially if and when it functions 
beyond powerful known influences on achievement, will be important for both developing 
theory and informing practice. 

The general research question for the study was: 

What are the effects of classroom assessment environment variables 
on^early achievement in mathematics and science in grades 7 through 

Specific questions implied by this general one included: How can these effects be 
modeled? What is the nature of the effects (direct, indirect, suppressive, etc.)? 
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Method 



Data source. The Longitudinal Study of American Youth (LSAY) used a national 
probability sample, stratified by geographic region and degree of urban development, of 
approximately 3000 students each in the 7th and 10th grades in the U.S. public schools in 
the fall of 1987. Students were followed for four years, until 1991. This study used data 
from both cohorts. Data were obtained from the LSAY project office, on computer tape. 
Codebooks provided information about the sample, instruments, and variables (Miller, 
Hoffer, Suchner, Brown, & Nelson, 1992a,b). 

Instrumentation. The LSAY surveys and tests did not include all Classroom 
Assessment Environment variables of theoretical interest. However, this study was 
designed because in its various surveys and tests, the LSAY included reasonable 
approximations to many of the variables of interest, for a nationally representative sample 
over four years of time. This afforded the opportunity to explore the effects of classroom 
assessment environment on achievement. Table 1 lists all variables, and the scales used 
to measure them, for this study. 

Outcomes of interest included mathematics and science achievement, each 
measured yearly on a 100-point scale developed from a 3-parameter IRT model. 
Background variables included gender (coded 0-1 ), SC:S (a composite variable of parental 
education, parental occupational status, and a household possession index), and general 
academic ability. The academic ability variable was operationally defined as scores on a 
15-item reading comprehension test (ot^.86) that has been found to function as a general 
academic measure in other LSAY analyses (Miller, 1992). 

For measures of the classroom assessment environment, variables from student 
and teacher questionnaires were inspected. The following variables were selected as the 
best indicators of classroom assessment environment available in the LSAY data because 
they measured either the frequency or quality of opportunities teachers provided for 
students to do work that would be judged or students' reactions to these opportunities. 
These classroom assessment environment variables were from teacher reports: percent 
of class time allotted to testing/ evaluation; frequency (scale 1-5, every day to very rarely) 
of written reports on experiments (science only), oral reports, design/conduct own science 
projects (science only), written reports on outside readings, and (for some years) explain 
reasoning to arrive at answer; and homework hours per week assigned, percent students 
completing homework on time, and percent homework collected and returned. These 
variables were from student reports: liking the subject matter, teacher clarity, challenge, 
perceived career utility, difficulty, and homework. Students were matched with the teacher 
variables from their respective mathenriatics or science classroom teachers for each year. 
The student variables provided measures of student perceptions of and reactions to the 
classroom assessment environment, hypothesized to be important mediators between the 
environment and performance (Pintrich & Schrauben, 1992; Schunk, 1994). 
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Analyses. Likert scales were reversed so that the more positive student responses 
(e.g., challenges a lot) and the teacher responses indicating more frequent use of a method 
(e.g., every day) were at the high end of the scales. All analyses used sampling weights. 
Path analysis was the major analytical tool, and the student was the unit of analysis. Most 
classes in the LSAY data reconfigured each year; path analysis maintained the longitudinal 
design for students. Four separate path analyses were done, in mathematics and science, 
respectively, for each cohort of students. Figures 1 and 2 present the original models 
tested. They differ in that Cohort 1 had three years (1 0th through 1 2th grades) and Cohort 
2 had four years (7th through 10th grades) of data. 

Stepwise regression was used to fit these recursive models. Using regression to 
fit models for successive years allowed for using the maximum number of students in each 
year, and each year's analysis used sampling weights for the respective years. Thus these 
models reflect the maximum available information. However, fitting the recursive model 
by means of separate regressions did not permit the calculation of an overall fit statistic for 
the entire model. It was reasoned that the inclusion of information from hundreds more 
subjects made this cost worthwhile. Table 2 presents weighted sample sizes and adjusted 
values for each of the stepwise regressions that contributed to the fitted models. Note 
that sample size varies widely because of availability of complete data. 

Results 

Tables 3 through 6 present the effects for the fitted models, which report the path 
coefficients from the best model identified for each of the achievement measures. 
Variables tested (see Table 1 ) but not listed in Tables 3 through 6 did not have statistically 
significant effects and were dropped from the model. The effects that were identified must 
be considered tentative findings because of the missing data problem for some of the 
component analyses. The missing data for high school seniors may be explained by the 
number of students who finish math and science requirements before their senior year and 
do not take additional classes in these subjects. The small numbers in some years for 
Cohort 2, especially in 7th ^rade science, present a problem for generalization. 

Math. The largest effects on ultimate math achievement, for both the junior high and 
high school cohorts, were the effects of prior achievement and general reading ability 
(Tables 3 and 4). SES and gender had greater effects on the achievement of the high 
school cohort than the junior high school cohort. The effect of gender was in favor of male 
students. 

Some of the classroom assessment environment (CAE) variables had statistically 
significant effects, too. For Cohort 1 , the percent of students completing homework on time 
in 10th grade math class and the hours of homework assigned per week in both 10th and 
1 2th grade math classes are each roughly a quarter of the size of the effect of reading 
ability. The percent of time devoted to testing and evaluation in 1 0th grade math class and 
the percent of homework corrected and returned to students had negative effects on 



achievement. 

For Cohort 2, the strongest positive CAE effects were for perceived teacher clarity 
and percent of students completing homework on time in 10th grade math class, percent 
completing homework on time in 7th grade math class, and perceived difficulty of 7th grade 
math class. Percent of homework corrected and returned to students in 7th, 8th, and 9th 
grades had negative effects on achievement. The percent of class time used for testing 
and evaluation had a negative valence in 8th grade, as for 10th grade in Cohort 1, but a 
positive valence in' 7th grade. How often written reports were assigned in 9th grade math 
class had a negative enGct on achievement. In general, for both cohorts, the homework 
variables were the CAE variables that remained in the models. 

Science. The largest effects on ultimate science achievement for both cohorts were 
the effects of prior science achievement and general reading ability (Tables 5 and 6). For 
both cohorty, there was a gender effect on achievement, favoring males. This effect was 
fairly large for the high school cohort. 

Some CAE variables also had statistically significant effects. For Cohort 1 , positive 
effects included percent completing homework on time in 10th grade science class, hours 
of homework assigned per week in 10th grade, how often science projects were assigned 
in 10th grade, and perceived difficulty of 10th grade science class. Negative effects were 
found for how often oral reports vv/ere assigned in 10th grade science class and the percent 
of homework corrected and returned to students in 11th grade science class. 

For Cohort 2, the strongest CAE effect was from how often science projects were 
assigned in 7th grade science class. Perceived clarity of the 7th grade textbook and how 
often written reports were assigned in 8th grade also had positive effects. Negative effects 
were found for how often written reports were assigned in 7th grade science class, how 
often oral reports were assigned in 8th grade, and how often students were called upon to 
explain their reasoning in 9th grade science class. 

Comparing math an J science. There are some similarities and some differences 
among the math and science models. The effects of prior achievement on subsequent 
achievement were strong for both math and science. So was the effect of reading ability, 
which was even stronger for Cohort 2 (grades 7-10) than for Cohort 1 (grades 10-12). 
There were SES effects for mathematics but not for science achievement. Gender effects 
ware stronger for science than for math, and they were stronger for the older cohort than 
for the younger (-.28 cf-.^2 for science; -.13 cf .00 for math). 

Some patterns are apparent for the CAE effects overall. Homework variables were 
most pervasive for mathematics achievement. Homework variables and the frequency of 
various types of assessment opportunities (oral reports, science projects) shared in the 
overall effect on achievement for science in high school. Frequency of various types of 
assessment opportunities (oral reports, written reports, sciences projects) were significant 



for science achievement in Cohort 2. 

The direction of the CAE effects is consistent for some variables and inconsistent 
for others. Percent of homework corrected and returned to students had a negative effect 
whenever it appeared. Hours of homework assigned and percent of students in the class 
completing homework on time had positive effects whenever they appeared. Percent of 
class time used for testing and evaluation, which had a significant effect only on math 
achievement, had mixed valences (2 negative and 1 positive). Frequency of oral reports, 
which had a significant effect only in science, had negative values. Frequency of written 
reports had mixed effects (3 negative and 1 positive). Frequency of science projects had 
a positive effect both times it appeared, once each for both the older and younger cohorts 
of students. 

Differences noted among the math and science models may reflect real differences 
in effects on math and science achievement. However, to the extent that findings were 
artifacts of reduction in sample size for some components of the analyses, apparent 
differences between effects on math and science achievement may be spurious. 

Discussion 

The following interpretations may be made of the results for the students whose 
achievement was analyzed in this study. Sample size problems limit the generalizability 
of results and preclude claims of national representativeness. 

Does this study support the importance of the classroom assessment environment 
for achievement? Was it amazing that there were CAE effects that were not washed out 
by the powerful effects of student background and prior achievement? Or was it 
disappointing that the effects weren't larger? 

The results do show that there were some CAE effects (Stiggins & Conklin, 1992) 
beyond student background and prior achievement. And CAE theory does explain some- 
-but not aII--of these effects. Most of the CAE variables tested that remained in the models 
{i.e., that had signficiant effects) were teacher report variables. This suggests that 
classroom controllabies are important and that an "environment" can indeed be created. 

The importance of homework variables to mathematics achievement is probably best 
explained by learning theory and the importance of practice to mastery, especially in 
solving the kind of formal problems posed in math. But it is in the classroom assessment 
environment that this student practice with the material is mediated (assigned, explained, 
feedback given, etc.), so the classroom assessment environment has a role to play. The 
surprising finding was the negative effect on achievement of percent of homework 
corrected and returned to students. This appears to be counter to the results that a theory 
of the classroom assessment environment would predict, namely, that homework corrected 
and returned would function as informative feedback and should affect achievement 



positively. The theory could be wrong. But since It is counter-Intuitive as well as counter 
to the theory to suggest that informative feedback is bad for achievement, consider some 
other possible explanations. Perhaps many of the classes where lots of homework was 
corrected and returned to students were classes in which exercises were corrected in 
class, round robin fashion, at the beginning of the next day, perhaps even in "switch with 
your neighbor" fashion. Or perhaps many of the classes where a large amount of 
homework was corrected and returned used student graders (not active self-assessors, but 
answer-checking clerical helpers). The negative effects of these practices on the 
classroom environment, especially in making students' mistakes public, could be expected 
to ruin the informative effects of the feedback. Classroom assessment environment theory 
would explain that, since student perceptions of their own competence and emotional 
factors (for example, embarrassment) would cause feedback from such corrected 
homework to have a "controlling" function, not an informational one (Brookhart, 1 995; Ryan, 
Connell, & Deci, 1985). 

The importance of the frequency of oral reports, written reports, and science projects 
on science achievement is also interesting for classroom assessment environment theory. 
The formats used for classroom assessments are an important part of the classroom 
assessment environment (Stiggins & Conklin, 1 992). The effect of these format variables 
for science and not mathematics is probably a function of subject matter. Science lends 
itself to oral and written reports and to projects; math lends itself to practice problems, and 
therefore to homework. 

Format effects were not all positive. Oral reports had negative effects on 
achievement. The simplest explanation for this may corne from learning theory and time 
on task. Oral reports take a great deal of class time, during which one student may be 
thinking hard while other students are much less engaged. Science projects had positive 
effects. This makes sense from several theoretical points of view: active learning, student 
construction, curiosity and motivation, self-evaluation. Written reports had mixed effects, 
as did percent of class time on testing and evaluation, presumably mostly paper-and- 
pencil tests. For each, there was one positive value. It is possible that these variables had 
negative effects, with a spurious positive finding. It is also possible the effects of these 
formats are truly mixed, depending on their use and place in the classroom assessment 
environment. 

The classroom assessment environment should play an important role in student 
achievement. But what role? Research in this area does not share the thirty-odd year 
tradition of research about other classroom instructional practices. This study joins a 
developing body of research in classroom assessment. Its unique contribution is that it is 
based on the LSAY. Most classroom assessment research samples classrooms from one 
or a few buildings or districts. Interpreting the relationships among variables across so 
many classrooms identified homework for math and assessment format foi science as 
particularly important to the classroom assessment environment. The siu Jy of these 
relationships was limitated to the variables available in the LSAY data. A further limitation 
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is the existence of specific context, culture, and personality effects that would not be 
represented in this kind of model. 

The general relationships discovered in these data could, however, serve as the 
basis for focused field studies. Alternative assessment formats in science, particularly, 
might be the focus of studies that included classroom time use in the investigation. The 
use of corrective feedback on math homework and other practice problems is another area 
for study. Understanding the place of these and other relationships between classroom 
assessment and achievement will be critical to the development of a more complete theory 
of classroom assessment than now exists. A theory of classroom assessment should 
guide current experimentation with alternative assessment and critiques of these and more 
traditional practices. The goal of classroom assessment is student achievement. The 
results of this study support the beginnings of a theory to explain how that happens. 
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Table 1 

Variables used in path analyses, scales, and reliabilities 



VARIABLE 



Background Variables 

• Gender 

• SES (composite of indexes of parent education, 

occupation.and household possessions) 

• Reading comprehension 

Achievement (measured each fall. 1987-1991) 

• Mathematics (based on IRT) 

• Science (based on IRT) 

Classroom Assessment Environment Variables 
Student Report (from fall surveys. 1987-1991) 

• like subject 

• teacher clarity 

• challenge of class 

• career utility 

• text clarity 

• difficulty 

• hours homework/week 

Teacher Report (from fall surveys. 1987-1991) 
Classroom work 

• percent class time on testing/evaluation 

• written reports on experiments or 

systematic observations (science only) 

• oral reports 

• design/conduct own science projects 

(science only) 

• written reports on outside readings 

• explain reasoning to arrive at answer 

(years 3-4 only) 
Homework 

• Hours per week assigned 

• Percent of students completing hw on time 

• Percent nw corrected & returned to students 



SCALE Refebiy 



0=M, 1=F 

z scores (-3 to +3) 

0-15 .80- .86 



0-100 .86-.95 
0-100 .79-.91 



1-5, Really like to Hate" 

1-5, Very clear to Not clear at all" 

1-5, Challenges a lot to Never challenges" 

1-5, Very useful to No use" 

1-5, Very clear to Hard to understand" 

1-5, Very easy to Very difficult" 

hours 



0- 100 

1- 5, Every day to Very rarely" 

1-5, Every day to Very rarely** 
1-5, Every day to Very rarely** 

1-5, Every day to Very rarely** 
1 -5, Every day to Very rarely** 



hours 
0-100 
0-100 



* The reliability values are from the LSAY codebook (Miller et al., 1 992). Reliability for background variables 
was calculated for both cohorts. Reliability for achievement variables and for classroom assessment 
environment variables was calculated each year for both cohorts. 

** Likerl scales were reversed for analysis. 



Table 2 



Weighted sample sizes and adjusted values for stepwise regression analyses used for 
path analysis 



Cohort One 
{10th-12th grades) 



Cohort Two 
(7th- 10th grades) 



Sample year 


n 


Math 


Science 
n R2 


Math 

n 




Science 
n R2 


1987 


946 


.37 


813 


.34 


1527 


.38 


173 .34 


1988 


739 


.73 


585 


.66 


1463 


.72 


1256 .69 


1989 


118 


.83 


159 


.68 


181 


.69 


417 .70 


1990 










989 


.67 


429 .64 
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Table 3 

Summary of effects on 1 2th grade mathematics achievement for Cohort One 



Effect Direct Indirect Total 




Gender -.13 -.13 

SES .08 .08 

Reading .43 .43 

Prior Math Achievement 

10th grade .48 .36 .84 

1 1 th grade .45 .45 

1Qt h CAE 

percent class time testing -.05 -.05 

hours homework assigned .09 .09 

percent completing hw on time .14 .14 

percent homework returned -.07 -.07 

11th CAE .00 

12th CAE 

hours homeworK assigned .09 .09 



CAE -- Classroom Assessment Environment 
hw - homework 
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Table 4 

Summary of effects on 1 0th grade mathematics achievement for Cohort Two 



Effect Direct Indirect Total 



Background 

SES .02 .02 

Reading .19 .38 .57 

Prior Math Achievement 

7th grade .21 .30 .51 

8th grade .22 .22 

9th grade .49 .49 

7th CAE 

difficulty .05 .05 

percent class time testing .03 .03 

how often written reports ■ -.03 -.03 

hours homework assigned .03 .03 

percent completing hw on time .09 .09 

percent homework returned -.03 -.03 

8th CAE 

percent class time testing -.01 -.01 

hours homework assigned .01 .01 

percent completing hw on time .01 .01 

percent homework returned -.01 -.01 

9th CAE 

how often written reports -.05 ••.05 

percent homework returned -.05 -.05 

10th CAE 

teacher clarity .06 .06 

percent completing hw on time .06 .06 



CAE -- Classroom Assessment Environmeni 
hw - homework 
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Table 5 

Summary of effects on I2th grade science achievement for Cohort One 



Effect Direct Indirect Total 



Background 

Gender -.11 -.17 -.28 

Reading .38 .38 

Prior Science Achievement 

10th grade .32 .38 .70 

11th grade .50 .50 

10th CAE 

difficulty .06 .06 

how often oral reports -.07 -.07 

how often science projects .06 .06 

hours homework assigned .06 .06 

percent completing hw on time .08 .08 

11th CAE 

percent homework returned -.03 -.03 

12th CAE .00 



CAE ~ Classroom Assessment Environment 
hw -- homework 
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Table 6 

Summary of effects on 1Qth grade science achievement for Cohort Two 



Effect Direct Indirect Total 

Background 

Gender -.12 -.12 

Reading .15 .37 .52 

Prior Math Achievement 

7th grade .51 .51 

8th grade .47 .13 .60 

9th grade .28 .28 

7th CAE 

text clarity .08 .08 

how often science projects .13 .13 

how often written reports -.08 -.08 

8 th CAE 

how often oral reports -.04 -.04 

how often written reports .03 .03 

9th CAE 

how often explain reasoning -.02 -.02 

10th CAE .00 



CAE -- Classroom Assessment Environment 
hw - homework 
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Figure 1 . Model for Mathematics and Science 
Achievement for Cohort 1 



I Backgrounds 

■Gender 
I -SES 
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\^ Ability / 
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StudentRept 
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Achievement / 
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V Achievement / 
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CAE 

StudentRept 
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/ CAE 

/ StudentRept \ 
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10th Grade 
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Two separate models were tested, for Mathematics and Science classes, respectively. 
The Math model used Classroom Assessment Environment (CAE) variables for Math 
classes and Mathematics achievement scores. The Science model used CAE 
variables for Science classes and Science achievement scores. 
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